1. Field of the Invention
The present invention relates to computer systems and, more particularly, to improved methods and apparatus for dynamically altering configuration of clustered computer systems.
2. Description of the Related Art
In contrast to single mainframe computing models of the past, more distributed computing models have recently evolved. One such distributed computing model is known as a clustered computing system. FIG. 1 illustrates an exemplary clustered computing system 100 including computing nodes (nodes) A, B and C, storage devices (e.g., storage disks 102-104), and other computing devices 106-110 representing other devices such as scanners, printers, digital cameras, etc. For example, each of the nodes A, B and C can be a computer with its own processor and memory. The collection of nodes A, B and C, storage disks 102-104, and other devices 106-110 make up the clustered computing system 100.
Typically, the nodes in a cluster are coupled together through a xe2x80x9cprivatexe2x80x9d interconnect with redundant pathways. As shown in FIG. 1, nodes A, B and C are coupled together through private communication channels 112 and 114. For example, the private communication channels 112 and 114 can adhere to Ethernet, ATM, or Scalable Coherent (SCI) standards. A client 116 can communicate with the clustered computing system 100 via a network 118 (e.g., public network) using a variety of protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), etc. From the point of view of the client 116, the clustered computing system 100 is a single entity that can provide the client 116 with a variety of computer-implemented services, e.g., web-hosting, transaction processing, etc. In other words, the client 116 is not aware of which particular node(s) of the clustered computing system 100 is (are) providing service to it.
The clustered computing system 100 provides a scalable and cost-efficient model where off-the-shelf computers can be used as nodes. The nodes in the clustered computing system 100 cooperate with each other to provide a distributed computing model that is transparent to users, e.g., the client 116. In addition, in comparison with single mainframe computing models, the clustered computing system 100 provides improved fault tolerance. For example, in case of a node failure within the clustered computing system 100, other nodes can take over to perform the services normally performed by the node that has failed.
Typically, nodes in the clustered computing system 100 send each other xe2x80x9cresponsivexe2x80x9d (often referred to as xe2x80x9cheart beatxe2x80x9d or activation) signals over the private communication channels 112 and 114. The responsive signals indicate whether nodes are active and responsive to other nodes in the clustered computing system 100. Accordingly, these responsive signals are periodically sent by each of the nodes so that if a node does not receive the responsive signal from another node within a certain amount a time, a node failure can be suspected. For example, in the clustered computing system 100, if nodes A and B do not receive a signal from node C within an allotted time, nodes A and B can suspect that node C has failed. In this case, if nodes A and B are still responsive to each other, a two-node sub-cluster (AB) results. From the perspective of the sub-cluster (AB), node C can be referred to as a xe2x80x9cnon-responsivexe2x80x9d node. If node C has really failed then it would be desirable for the two-node sub-cluster (AB) to take over services from node C. However, if node C has not really failed, taking over the services performed by node C could have dire consequences. For example, if node C is performing write operations to the disk 104 and node B takes over the same write operations while node C is still operational, data corruption can result.
It should be noted that the fact that nodes A and B have not received responsive signals from node C does not necessarily mean that node C is not operational with respect to the services that are provided by node C. Other events can account for why responsive signals for node C have not been received by nodes A and B. For example, the private communication channels 112 and 114 may have failed. It is also possible that node C""s program for sending responsive signals may have failed but node C is fully operational with respect to the services that it provides. Thus, it is possible for the clustered computing system 100 to get divided into two or more functional sub-clusters wherein the sub-clusters are not responsive to each other. This situation can be referred to as a xe2x80x9cpartition in spacexe2x80x9d or xe2x80x9csplit brainxe2x80x9d where the cluster no longer behaves as a single cohesive entity. In this and other situations, when the clustered computing system no longer behaves as a single cohesive entity, it can be said that the xe2x80x9cintegrityxe2x80x9d of the system has been compromised.
In addition to partitions in space, there are other potential problems that need to be addressed in managing the operation of clustered computing systems. For example, another potential problem associated with operating clustered computing systems is referred to as a xe2x80x9cpartition in timexe2x80x9d or xe2x80x9camnesia.xe2x80x9d As is known to those skilled in the art, partitions in time can occur when a clustered computing system is operated with cluster configurations that vary over time.
One problem is that the conventional methods do not provide for techniques that allow alteration of configuration of the clustered computing systems dynamically. For example, adding a new node to a clustered computing system typically requires shutting down all the existing nodes in the clustered computing system in order to guard against undesired partitions in time or space. Similarly, removing a node typically requires shutdown of all other existing nodes in the clustered computing system.
In view of the foregoing, there is a need for techniques that enable dynamic configuration changes to clustered computing systems.
Broadly speaking, the invention relates to improved techniques for dynamically altering configurations of clustered computing systems. In one aspect, the improved techniques allow alteration of an existing configuration of a clustered computing system without having to completely shutdown the clustered computing system. Accordingly, components such as nodes or other devices (e.g., peripheral devices) can be added to, or removed from, the clustered computing system while one or more existing nodes remain active. As a result, the clustered computing system can provide uninterrupted services while the configuration of the clustered computing system is being dynamically altered, yet also safeguard against unwanted partitions in time or space.
The invention can be implemented in numerous ways, including a system, an apparatus, a method or a computer readable medium. Several embodiments of the invention are discussed below.
As a method for altering configuration of a clustered computing system, one embodiment of the invention includes the acts of: identifying a first component that is to be added to or removed from the clustered computing system, and updating component vote information associated with at least one active component of the clustered computing system while the at least one active component remains active.
As a method for altering configuration of a clustered computing system having at least one active component with associated configuration vote information, another embodiment of the invention includes the acts of: receiving a configuration alteration request for addition or removal of one or more components to or from the existing configuration of the clustered computing system; selecting one of the components associated with the configuration alteration request as a selected component; obtaining a vote for the selected component; updating the configuration vote information of the active component in accordance with the vote while the at least one active component remains active; determining whether the updating of the configuration vote was successful; and determining whether there are other components associated with the configuration alteration request to be selected. When other components are to be selected, the method can operate to add or remove the other components.
As a clustered computing system, an embodiment of the invention includes a computing cluster including at least one computing node, and a configuration manager provided for the at least one computing node to update component vote information associated with at least one active component of the clustered computing system while the at least one active component remains active.
As a computer readable media including computer program code for altering configuration of a clustered computing system having at least one active component with associated configuration vote information, an embodiment of the invention includes: computer program code for receiving a configuration alteration request, the configuration alteration request requesting addition or removal of one or more components to or from the existing configuration of the clustered computing system; computer program code for selecting one of the components associated with the configuration alteration request as a selected component; computer program code for obtaining a vote for the selected component; computer program code for updating the configuration vote information of the active component while the at least one active component remains active; computer program code for determining whether the computer program code for updating the configuration vote information has successfully updated the configuration vote information; and computer program code for determining whether there is another component associated with the configuration alteration request to be selected.
As computer readable media including computer program code for altering configuration of a clustered computing system including at least one component, an embodiment of the invention includes: computer program code for identifying a first component that is to be added to or removed from the clustered computing system; and computer program code for updating component vote information associated with at least one active component of the clustered computing system while the at least one active component remains active.
The invention has numerous advantages. One advantage is that the invention provides for dynamic alteration of configurations of clustered computing systems. Another advantage is that dynamic alterations can be achieved without causing unwanted partitions in time or space. Still another advantage is that the techniques of the invention can be implemented without having to substantially interrupt the operations and services provided by the clustered computing systems.